Skip to content

wc: avoid pipe() if input is pipe#11636

Merged
sylvestre merged 1 commit intouutils:mainfrom
oech3:wc-omit-pipe
Apr 20, 2026
Merged

wc: avoid pipe() if input is pipe#11636
sylvestre merged 1 commit intouutils:mainfrom
oech3:wc-omit-pipe

Conversation

@oech3
Copy link
Copy Markdown
Contributor

@oech3 oech3 commented Apr 4, 2026 •
edited
Loading

Avoid unnecessary pipe() call (1 MiB RAM usage). This avoids performance drop on the system opening too many pipes too:

$ time cat h| strace -o /dev/null -e inject=pipe:error=ENOENT target/release/wc -c
1000000000
Executed in   51.41 millis    fish           external
   usr time   16.81 millis    1.53 millis   15.27 millis
   sys time   60.23 millis    1.40 millis   58.83 millis

$ time cat h| strace -o /dev/null -e inject=pipe:error=ENOENT wc -c
1000000000
Executed in  258.63 millis    fish           external
   usr time   60.48 millis    0.00 millis   60.48 millis
   sys time  242.36 millis    3.15 millis  239.21 millis

(benchmark result is not important at here)

@oech3
Copy link
Copy Markdown
Contributor Author

oech3 commented Apr 4, 2026 •
edited
Loading

I think we need a helper function to deduplicate the logic to force-splice() via additional pipe before introducing fast-path to many utilities.
I don't like current splice_exact function, but it seems current code is optimal...

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 4, 2026

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/basenc/bounded-memory is now passing!
Congrats! The gnu test tests/rm/many-dir-entries-vs-OOM is now passing!
Congrats! The gnu test tests/tail/tail-n0f is now passing!

@oech3 oech3 marked this pull request as ready for review April 4, 2026 13:28
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 5, 2026

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/resolution (passes in this run but fails in the 'main' branch)

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 6, 2026

GNU testsuite comparison:

Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/printf/printf-surprise is now being skipped but was previously passing.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 7, 2026

GNU testsuite comparison:

GNU test failed: tests/misc/io-errors. tests/misc/io-errors is passing on 'main'. Maybe you have to rebase?
GNU test failed: tests/timeout/timeout-group. tests/timeout/timeout-group is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/printf/printf-surprise is now passing!

@oech3 oech3 mentioned this pull request Apr 7, 2026
Merged
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 7, 2026

GNU testsuite comparison:

Skipping an intermittent issue tests/cut/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/basenc/bounded-memory is now being skipped but was previously passing.

@sylvestre sylvestre merged commit c48b5fe into uutils:main Apr 20, 2026
169 checks passed
@oech3 oech3 deleted the wc-omit-pipe branch April 20, 2026 21:26
@oech3
Copy link
Copy Markdown
Contributor Author

oech3 commented Apr 20, 2026 •
edited
Loading

I think this code is used only with pipe input:

$ time strace -c -e pipe wc -c </tmp/huge
1000000000
________________________________________________________
Executed in  151.21 millis    fish           external
   usr time   20.46 millis    0.00 millis   20.46 millis
   sys time  123.15 millis    1.46 millis  121.69 millis

$ time cat /tmp/huge | strace -c -e pipe wc -c
1000000000
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00    0.000007           7         1           pipe
------ ----------- ----------- --------- --------- ----------------
100.00    0.000007           7         1           total
________________________________________________________
Executed in   64.75 millis    fish           external
   usr time   16.98 millis    0.86 millis   16.12 millis
   sys time   61.64 millis    1.89 millis   59.75 millis

(using wc before this PR)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@oech3 @sylvestre